PR Summary

This pull request introduces several enhancements and bug fixes to the ValidMind Library, particularly focusing on credit risk scorecard modeling. The key changes include:

1. New Notebooks: Two new Jupyter notebooks have been added to demonstrate the application scorecard model using the ValidMind Library. These notebooks provide a step-by-step guide for loading a demo dataset, preprocessing data, training models, and documenting the model using ValidMind.

2. New Tests: Several new tests have been added to the validmind/tests directory, including:

   • MutualInformation: Evaluates feature relevance by calculating mutual information scores between features and the target variable.
   • ScoreBandDefaultRates: Analyzes default rates and population distribution across credit score bands.
   • CalibrationCurve: Assesses the calibration of probability estimates by comparing predicted probabilities against observed frequencies.
   • ClassifierThresholdOptimization: Analyzes and visualizes different threshold optimization methods for binary classification models.
   • ModelParameters: Extracts and displays model parameters for transparency and reproducibility.
   • ScoreProbabilityAlignment: Evaluates the alignment between credit scores and predicted probabilities.

3. Enhancements to Existing Tests: Modifications have been made to existing tests to improve their functionality and accuracy. For example, the TooManyZeroValues test now includes a row count and uses a percentage threshold for zero values.

4. Dataset Splitting Functionality: The split function in lending_club.py has been enhanced to support an optional validation set, allowing for more flexible dataset splitting.

5. Test Configuration Utility: A new utility function get_demo_test_config has been added to generate a default test configuration for demo purposes.

6. Version Update: The version of the ValidMind Library has been updated from 2.7.3 to 2.7.4.

7. Bug Fixes: Various bug fixes have been implemented, including corrections to test logic and improvements to test coverage.

Test Suggestions

• Run the new Jupyter notebooks to ensure they execute without errors and produce the expected outputs.
• Verify the functionality of the new tests by running them with different datasets and configurations.
• Test the enhanced split function with various dataset sizes and configurations to ensure it correctly handles train, validation, and test splits.
• Check the accuracy and performance of the MutualInformation and ScoreBandDefaultRates tests with known datasets.
• Validate the CalibrationCurve and ClassifierThresholdOptimization tests by comparing their outputs with expected calibration and threshold optimization results.
• Ensure the ModelParameters test correctly extracts parameters from different model types.
• Test the ScoreProbabilityAlignment test with datasets having different score distributions.